Nonasymptotic Analysis of Monte Carlo Tree Search

In “Nonasymptotic Analysis of Monte Carlo Tree Search,” D. Shah, Q. Xie, and Z. Xu consider the popular tree-based search strategy, Search (MCTS), in context infinite-horizon discounted Markov decision process. They show that MCTS with an appropriate polynomial rather than logarithmic bonus term indeed leads to desired convergence property. The authors derive results by establishing a concentration property regret for class nonstationary multiarm bandits. Furthermore, using this as building block, they demonstrate MCTS, combined nearest neighbor supervised learning, acts “policy improvement” operator can iteratively improve value function approximation.